Arrangement for distributing head related transfer function filters

ABSTRACT

Arrangement for distributing head related transfer function filters. In the arrangement a user device sends a request for a head related transfer function filter to the service being used. The service verifies if the user of the device has a subscription for a head related transfer function filters in the service being used and retrieves a filter as a response to a positive verification result. The service may filter audio channels and transmit filtered audio further. In an alternative embodiment the service transmits the filter to the user device for filtering the audio.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 16/516,036, filed Jul. 18, 2019, which claims priority toFinnish Patent Application No. 20185678, filed Aug. 13, 2018, and theseapplications are specifically incorporated by reference herein in theirentirety.

BACKGROUND

Audio systems with multiple audio channels are commonly known and usedby entertainment industry, for example in movies or computer games.These systems are often called Surround Sound systems orthree-dimensional sound systems. Recently arrangements for implementingeven better three-dimensional sound experience have been introduced.These arrangements do not only have multiple audio channels but provideobject-based audio for improving the listening experience.

In conventional approaches each of the multiple audio channels areprovided to the user of the sound system. Typically audio channels arereceived at an amplifier or similar device that distributes the audiochannel to respective speakers that have been connected to theamplifier. The amplifier may modify the received audio channels so thatthe playback is adjusted to the number and locations of speakers.

Recently arrangements for providing similar experience using headphoneshave been developed. Typically in headphone listening these arrangementsare based on filtering the sound channels using so called head relatedtransfer function filters. The three-dimensional experience is producedby manipulating sounds in headphones two audio channels so that theyresemble directional sounds arriving at the ear-canal. Thethree-dimensional sound experience is possible by taking into accountthe effect of pinna, head and torso to the sound that enters to theear-canal. These filters are often called HRTF-filters (Head-RelatedTransfer Function). These filters are used to provide an effect that issimilar to how a human experiences sounds arriving from differentdirections and distances. When the anatomy of body parts, such as ears,head and torso, of a person is known a personal HRTF-filter can beproduced so that the sound experienced through headphones is asrealistic as possible.

The field of providing realistic audio systems is under constantdevelopment. Thus, there is a need for improvements both in audioquality and efficient distribution of the audio.

SUMMARY

Arrangement for distributing head related transfer function filters. Inthe arrangement a user device sends a request for a head relatedtransfer function filter to the service being used. The service verifiesif the user of the device has a subscription for a head related transferfunction filters in the service being used and retrieves a filter as aresponse to a positive verification result. The service may filter audiochannels and transmit filter audio further. In an alternative embodimentthe service transmits the filter to the user device for filtering theaudio.

In an aspect a method for distributing a head related transfer functionfilter is disclosed. In the method first a request for a service from auser device is received. Then, the availability a subscription of aHRTF-filter for the requested service is determined. If the service isavailable, the method further comprises sending a request for a filterto a filter server. As a response to the sent request a filter from afilter server is received. The filter is a head related transferfunction filter that is designed for the user and possibly also for thedevice, audio format or the listening setup that the user is using.

In one embodiment the method further comprises filtering audio channelusing the received filter and transmitting the filtered audio channel tothe requesting user device. In another embodiment the method furthercomprises transmitting the received filter to the requesting user devicefor filtering the audio channel at the requesting device.

It is beneficial to implement the retrieval of the filter in the serveror service the user actually is wishing to use. This enables possibilityfor the service to check if the user has enabled the head relatedtransfer filter in the particular service and a correct filter can beretrieved based on the service information. Furthermore, when servicefilters audio channels there is no need for transmitting original audiochannels and transmission bandwidth can be saved.

In an implementation the filtering comprises filtering a number of audiochannels into a smaller number of audio channels. This is beneficial asit reduces bandwidth.

In an implementation the smaller number of audio channels is two. Thisis beneficial particularly when the user is using headphones with onlytwo channels. This provides a possibility to use three-dimensional,possibly with multiple sound sources, audio channels with bandwidth oftwo channels.

In an implementation the request for a filter comprises at least one ofthe following: user identification, application identification and userdevice identification. It is beneficial that the filter can be chosen byusing all parameters that can have an effect to audio production.

In an implementation the user device comprises headphones. It isbeneficial that the filter can be generated to specific headphone modelsso that they take into account differences in frequency response betweendifferent headphone models.

In an aspect a computer program for a server is disclosed. The computerprogram comprises code adapted to cause the method as disclosed above.

In another aspect an apparatus is disclosed. The apparatus comprises atleast one processor configured to execute computer programs; at leastone memory configured to store computer programs and related data; andat least one data communication interface (113) configured tocommunicate with external data communication networks. The apparatus isconfigured to perform a method as disclosed above. The method may becaused by a computer program as disclosed above.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of an arrangement for distributing head related transferfunction filters and constitute a part of this specification, illustrateembodiments and together with the description help to explain theprinciples of the arrangement for distributing head related transferfunction filters. In the drawings:

FIG. 1 is an example of an apparatus in an example system fordistributing head related transfer function filters,

FIG. 2 is an example of a method for distributing head related transferfunction filters, and

FIG. 3 is another example of a method for distributing head relatedtransfer function filters.

DETAILED DESCRIPTION

Reference will now be made in detail to the embodiments, examples ofwhich are illustrated in the accompanying drawings.

In the following examples of FIGS. 1-3 the most common application ofheadphone listening is described. However, similar principles may beused in filtering sound for different loudspeaker arrangements.

In FIG. 1 an example of a system comprising an apparatus 100 configuredto distribute head related transfer function filters, an apparatus 110configured to provide a streaming service and an end user apparatus 120configured to receive and repeat the stream.

The apparatus 100 is typically operated by a service provider and may beimplemented in a form of a server, cloud computing resource, database orsimilar to which requests for retrieving a pre-stored head relatedtransfer function filter.

The apparatus 100 comprises at least one processor 101, at least onememory 102 and at least one network connection 103. The at least onememory is may comprise volatile and non-volatile memories. Typically itincludes at least one non-volatile memory to which head related transferfunction filters belonging to different users are stored. The at leastone memory 102 may include a database and it is not necessary physicallyin the one and same device but may take a form of several differentcomputing units that form one logical service or computing resource.

The apparatus 100 is configured to receive requests through a networkconnection 103. The network connection may be any commonly knowninterface to data communication networks, such as Internet.

The requests that are received through the network connection 103 areprocessed by the at least one processor 101. The request is processed sothat from the request an identification of a head related transferfunction filter is extracted and the corresponding head related transferfunction filter is retrieved and provided as a response to the requestusing the at least one network connection 103.

The identification identifying the head related transfer function filtermay be unique to a user or even unique to different devices or devicemodels that have been assigned to a certain persons. Thus, it ispossible that one person has several head related transfer functionfilters stored and in addition to the user the request identifies thedevice to which the user is retrieving the head related transferfunction filter.

The apparatus 110 configured to provide a streaming service, such as amusic or movie service or any other service that includes providing anaudio stream with or without other information. The apparatus 110comprises at least one processor 111, at least one memory 112 and atleast one network connection. The at least one memory 112 may include adatabase and it is not necessary physically in the one and same devicebut may take a form of several different computing units that form onelogical service or computing resource.

The apparatus 110 receives service requests through the networkconnection 113 and processes them using the at least one processor 111.Based on the request the apparatus 110 extracts an identification for ahead related transfer function filter from the service request andproduces a request including the identification. The produced request isthen sent to the apparatus 110. As a response to the request theapparatus 110 receives a head related transfer function filter from theapparatus 100 that has provided it as a response to the request. Thereceived head related transfer function filter is store in the at leastone memory 112.

After receiving the head related transfer function filter the apparatus110 provides the request streaming service to the end user apparatus 120that has requested the streaming service. The service stream is modifiedfrom the standard stream so that the apparatus 110 processes using theat least one processor 111 the audio stream, which is possibleassociated with additional information. For example, the audio streammay be a sound track of a movie that is played together with the videostream of the movie. When processing the audio stream, the apparatus 110is configured to use the received head related transfer function filterto form a filtered audio stream that comprises typically two channels.This reduces the need for data transmission bandwidth and also removesthe need for an end user device that is capable of modify the audiostream in accordance with the head related transfer function.Furthermore, the need for computing power at the end user device isreduced when the demanding computations are done at the service providerside.

The end user apparatus 120 is typically a computer, tablet device suchas iPad or a mobile phone to which headphones 125 have been attached to.The end user apparatus comprises at least one processor 121, at leastone memory 122, at least one network connection 123 and at least oneaudio device 124 that is capable of producing sounds using headphones125. The end user sends a request for a streaming service to theapparatus 110 and as a response receives a stream transmitted by theapparatus 120. The at least one processor 121 processes the receivedstream that typically requires to be decoded. Then the audio part of thestream is transmitted to the audio device 124 that provides the audiostream in suitable format to the headphones 125.

In the arrangement described above the information of the availabilityof a head related transfer function may be included in the subscriptioninformation that is typically stored at the apparatus 110. The initialrequest received from the end user device may have further informationabout which head related transfer function filter is received if any.This facilitates possibility to share the subscription, for example,within a family.

In FIG. 2 an example method is illustrated. The method of FIG. 2 isimplemented in a streaming service or similar. Typically this kind ofstreaming service is implemented using a computing resource comprising anumber of servers that operate together for providing the service to alarge number of clients. In the following description the service isreferred as a one server, however, this should be understood that it maybe, and in many cases is, a virtual server that looks like one serviceto the client.

The method at the server is initiated when the server, such as theapparatus 110 of FIG. 1, receives a request for streaming service froman end user device, such as the apparatus 120, step 200. The initiationmay include several tasks, such as authentication of the user,determining the subscribed service level and choosing a stream, such asa movie, to be streamed. Furthermore, the request may includeinformation with regard the service level the user is wishing to have,such as the preferred resolution or sound quality.

The determination of service level may relate to checking allowed numberof simultaneous streams, image quality or audio quality. The determiningof used audio quality includes determination of subscription of headrelated transfer function filter service, step 201. This information maybe stored into the user information at the streaming service or it maybe requested from a server providing the actual head related transferfunction filter or both. It is possible to implement also alternativemethods for checking the availability of the subscription may be used.

If the user has indicated that he is wishing to use headphones withaudio track enhanced using head related transfer function filters andthe person has an appropriate subscription for the service, thestreaming server sends a request for a head related transfer functionfilter to a head related transfer function filter service, such as theapparatus 100 of FIG. 100, step 202. The request may include, forexample, the identification of the person wishing to have a head relatedtransfer function filter and the identification of the headphones beingused. In addition to identification of headphones used also type of theapplication, audio format being used and similar may be identified usingsimilar identification. It is noted, that the request may containrequest for several people if a plurality of persons are wishing toreceive the same stream simultaneously at the same venue. This happens,for example, when two persons are watching the same movie.

As a response to the request a head related transfer function filter isreceived, step 203. The filter, or a plurality of filters, is stored tothe streaming service for later use. The filter is then used inprocessing the audio track of the stream, step 204. Modem movies mayhave a very high number of audio channels or audio sources. Thestreaming service processes the audio track into two channels ofheadphones using the received head related transfer function, step 204.This reduces the required bandwidth for transmitting the audio track. Itis noted that even if the streaming service produces individual audiotracks for a plurality of users the bandwidth may still be saved as onlytwo channels per person are needed instead of the typically high numberof audio channels and sources that are available in modem movies.

Finally the streaming server transmits the stream to the user or users,step 205. Users receive their customized audio tracks at theirheadphones. The sound produced at the headphones is realistic andprovides three-dimensional feeling of the audio scene. This increasesthe user experience by facilitating experiencing the produced audio asit was produced by high number of speakers or experienced in live.

The example disclosed in FIGS. 1 and 2 is providing a streaming servicethat includes modifying the content of transmitted signals in a mannerthat as a result the physical signal sent to the final user is differentto arrangements where audio signals are not modified.

In FIG. 3 another example of a method that can be used in an arrangementsimilar to the arrangement of FIG. 1 is disclosed. The method of theexample is initiated by receiving a request for a game session, step300. The game session differs as an application from streaming servicesas it typically requires faster, preferably real-time, to game eventsand user movements.

The availability of the HRTF-subscription, at step 301, may bedetermined as it was determined for streaming services. Based on theavailability information a request for a filter is generated, step 302.In this example the request for a filter comprises additionalinformation. This additional information may be, for example, adescription of the space where the user of the game is currently locatedso that the filter will be able to produce audio that fits to the space.The request may be, and typically is, formed in view of the requestingapplication, which is a game in this context. Thus, the requestcomprises information about the user for verifying the subscription andalso for retrieving the body metrics that are useful in generating orselecting the requested filter.

Then the request is sent to a filter service, step 303. The filterservice may select a filter that has been produced earlier for the useror generate a filter specifically matching with the request. Then thefilter is received at the requesting device, step 304. Then, the filteris transmitted further to the device the user is using, step 305. Thisstep may include additional steps, such as, switching the audio mode ofthe game into a specific mode that takes the filter at the user end intoaccount.

Finally, the user receives at the user device the filter and startsapplying it to the audio track belonging to the game, step 306. The userdevice and the filter now produce and audio experience in a manner thatthe game server is assuming by modifying the audio tracks so that theactual sound signals are modified and thus different when compared to anaudio track without filter.

The examples above are focusing on a single person using headphones or aspace where other people are not disturbed. However, similar principlesmay be used using loudspeaker arrangements for a group of people whereone or more devices are using the same loudspeaker arrangement. Forexample, in a car it is possible that the music is played to all peoplebut navigation instructions are filtered so that they can be heard atthe driver's seat but in a manner the listening to the music is notdisturbed for other people. In such case, the car is the user device andit is typically connected to a server, such as car manufacturer'sserver, and it may request one or more filters, for example, based onthe occupancy of the car.

As stated above, the components of the exemplary embodiments can includecomputer readable medium or memories for holding instructions programmedaccording to the teachings of the present inventions and for holdingdata structures, tables, records, and/or other data described herein.Computer readable medium can include any suitable medium thatparticipates in providing instructions to a processor for execution.Common forms of computer-readable media can include, for example, afloppy disk, a flexible disk, hard disk, magnetic tape, any othersuitable magnetic medium, a CD-ROM, CD±R, CD±RW, DVD, DVD-RAM, DVD±RW,DVD±R, HD DVD, HD DVD-R, HD DVD-RW, HD DVD-RAM, Blu-ray Disc, any othersuitable optical medium, a RAM, a PROM, and EPROM, a FLASH-EPROM, anyother suitable memory chip or cartridge, or any other suitable mediumfrom which a computer can read.

It is obvious to a person skilled in the art that with the advancementof technology, the basic idea of an arrangement for distributing headrelated transfer function filters may be implemented in various ways.The arrangement for distributing head related transfer function filtersand its embodiments are thus not limited to the examples describedabove; instead they may vary within the scope of the claims.

What is claimed is:
 1. A method for distributing a head related transferfunction (HRTF) filter comprising: receiving a request for an audiosession from a user device coupled to headphones; determining whether auser associated with the user device has access to a HRTF filterservice; obtaining, in response to determining that the user has access,a headphone-specific HRTF filter based on an identification of the userand a model of the headphones, wherein the headphone-specific HRTFfilter takes into account a frequency response of the model of theheadphones; and transmitting the headphone-specific HRTF filter to theuser device for filtering an audio channel at the user device.
 2. Themethod of claim 1, wherein determining whether the user has accessincludes determining whether the user has a subscription to the HRTFfilter service.
 3. The method of claim 2, wherein obtaining theheadphone-specific HRTF filter includes sending, in response todetermining that the user has the subscription, a request for theheadphone-specific HRTF filter to a filter server, wherein the requestfor the headphone-specific HRTF filter comprises the identification ofthe user and the model of the headphones, and receiving theheadphone-specific HRTF filter from the filter server, wherein theheadphone-specific HRTF filter is unique to the user.
 4. The method ofclaim 1, wherein the request identifies a type of an applicationrequesting the audio session, and wherein obtaining the HRTF filter isbased on the type of the application.
 5. The method of claim 1, whereinthe request is for a game session, and further comprising: determiningthat the request is from a game application; and switching an audio modeof a game service to account for the headphone-specific filter beingused to filter the audio channel at the user device.
 6. The method ofclaim 1, wherein the user device is coupled to second headphonesassociated with a second user, and further comprising: obtaining asecond headphone-specific HRTF filter based on a second identificationof the second user and a second model of the second headphones, whereinthe second headphone-specific HRTF filter takes into account a secondfrequency response of the second headphones, and transmitting the secondheadphone-specific HRTF filter to the user device for filtering theaudio channel at the user device.
 7. The method of claim 1 furthercomprising: extracting the identification of the user and the model ofthe headphones from the request for the audio session; and generating arequest for the headphone-specific HRTF filter including the extractedidentification.
 8. An apparatus for distributing a head related transferfunction (HRTF) filter comprising: a data communication interfaceconfigured to communicate with a user device; a memory configured tostore instructions; a processor configured to execute the instructionsto cause the apparatus to receive a request for an audio session fromthe user device coupled to headphones, determine whether a userassociated with the user device has access to a HRTF filter service,obtain, in response to determining that the user has access, aheadphone-specific HRTF filter based on an identification of the userand a model of the headphones, wherein the headphone-specific HRTFfilter takes into account a frequency response of the model of theheadphones, and transmit the headphone-specific HRTF filter to the userdevice for filtering an audio channel at the user device.
 9. Theapparatus of claim 8, wherein determining whether the user has accessincludes determining whether the user has a subscription to the HRTFfilter service.
 10. The apparatus of claim 9, wherein obtaining theheadphone-specific HRTF filter includes sending, in response todetermining that the user has the subscription, a request for theheadphone-specific HRTF filter to a filter server, wherein the requestfor the headphone-specific HRTF filter comprises the identification ofthe user and the model of the headphones, and receiving theheadphone-specific HRTF filter from the filter server, wherein theheadphone-specific HRTF filter is unique to the user.
 11. The apparatusof claim 8, wherein the request identifies a type of an applicationrequesting the audio session, and wherein obtaining the HRTF filter isbased on the type of the application.
 12. The apparatus of claim 8,wherein the request is for a game session, and wherein the processor isconfigured to execute the instructions to cause the apparatus to:determining that the request is from a game application; and switchingan audio mode of a game service to account for the headphone-specificfilter being used to filter the audio channel at the user device. 13.The apparatus of claim 8, wherein the user device is coupled to secondheadphones associated with a second user, and wherein the processor isconfigured to execute the instructions to cause the apparatus to:obtaining a second headphone-specific HRTF filter based on a secondidentification of the second user and a second model of the secondheadphones, wherein the second headphone-specific HRTF filter takes intoaccount a second frequency response of the second headphones, andtransmitting the second headphone-specific HRTF filter to the userdevice for filtering the audio channel at the user device.
 14. Theapparatus of claim 8, wherein the processor is configured to execute theinstructions to cause the apparatus to: extract the identification ofthe user and the model of the headphones from the request for the audiosession; and generate a request for the headphone-specific HRTF filterincluding the extracted identification.
 15. A non-transitory computerreadable medium storing instructions, which when executed by one or moreprocessors of a data processing system, cause the data processing systemto perform a method comprising: receive a request for an audio sessionfrom a user device coupled to headphones; determine whether a userassociated with the user device has access to a HRTF filter service;obtain, in response to determining that the user has access, aheadphone-specific HRTF filter based on an identification of the userand a model of the model of the headphones, wherein theheadphone-specific HRTF filter takes into account a frequency responseof the headphones; and transmit the headphone-specific HRTF filter tothe user device for filtering an audio channel at the user device. 16.The non-transitory computer readable medium of claim 15, whereindetermining whether the user has access includes determining whether theuser has a subscription to the HRTF filter service.
 17. Thenon-transitory computer readable medium of claim 16, wherein obtainingthe headphone-specific HRTF filter includes sending, in response todetermining that the user has the subscription, the request for theheadphone-specific HRTF filter to a filter server, wherein a request forthe headphone-specific HRTF filter comprises identification of the userand the model of the headphones, and receiving the headphone-specificHRTF filter from the filter server, wherein the headphone-specific HRTFfilter is unique to the user.
 18. The non-transitory computer readablemedium of claim 15, wherein the request identifies a type of anapplication requesting the audio session, and wherein obtaining the HRTFfilter is based on the type of the application.
 19. The non-transitorycomputer readable medium of claim 15, wherein the request is for a gamesession, and wherein the method further comprises: determining that therequest is from a game application; and switching an audio mode of agame service to account for the headphone-specific filter being used tofilter the audio channel at the user device.
 20. The non-transitorycomputer readable medium of claim 15, wherein the user device is coupledto second headphones associated with a second user, and wherein themethod further comprises: obtaining a second headphone-specific HRTFfilter based on a second identification of the second user and a secondmodel of the second headphones, wherein the second headphone-specificHRTF filter takes into account a second frequency response of the secondheadphones, and transmitting the second headphone-specific HRTF filterto the user device for filtering the audio channel at the user device.